1. Field of the Invention
The present invention relates to a neural network for fuzzy reasoning, and more particularly to a multi-layered neural network which is trained based on a back-propagation algorithm.
2. Description of the Related Art
In a conventional multi-layered neural network, a weight of connection between neurons can be determined to decrease the error of the neural network based on a back-propagation algorithm by supplying the relation between the input and output data as a training data to the network. A nonlinear input/output relationship can be easily and precisely implemented by the multi-layered neural network. The conventional multi-layered neural network has merits of easy training, easy implementation of nonlinear input/output relationship, and precise workability. On the other hand, the conventional multi-layered neural network has the following technical problems.
(1) It is difficult for the operator to fully understand the internal expression of the neural network which is determined by means of training, and thus, the internal expression of the neural network cannot be effectively used as knowledge. PA1 (2) Based on the same reason as in the first problem, it is difficult to initially provide the neural network with human knowledge to set up schematic input/output relationship. Therefore, the neural network system cannot start training from the initial state representing roughly determined by the operator in order to shorten the training period. PA1 (3) Based on the same reason as in the first problem, in the event that a partial variation of the input/output relationship or modification of this relationship is needed, it is not possible to use the human knowledge to change the trained internal expression of the neural network. PA1 (1) Tatsumi FURUYA et al., "NFS: Fuzzy Inference System using a Neural Network", the transactions of the institute of information processing engineers of Japan, Vol. 30, No. 6, June 1989, pages 795 to 798. The proposed fuzzy inference system incorporates an if-part section and a then-part section which are respectively formed of neural networks, and outputs from both sections are combined with each other with a minimum operator. In place of using the membership function, in this system, the input and output are divided into a certain number of domains. Therefore, although being free from restrictive factors of the membership function, the proposed system cannot clearly express the knowledge of the input and output being "big" or "small". Furthermore, the fuzzy inference system is divided into plural networks for the if-part and then-part sections and training is performed for each section. It is difficult to set the input and output data properly for each section. In order to properly determine the input and output data for each section, the operator is obliged to perform a trial and error method. PA1 (a) It is desired that the neural network clearly expresses human knowledge. PA1 (b) It is desired to form the fuzzy inference system with a single network and simultaneously train the if-part section and the then-part section. PA1 (2) Isao HAYASHI et al., "Formulation of Fuzzy Reasoning by Neural Network", transactions of 4th fuzzy system symposium (Tokyo, May 30-31, 1988). Unlike the fuzzy inference system described in the first document, this proposed fuzzy inference system directly uses the input and output data without dividing them into a certain number of domains. In this system, the neural networks are provided for the if-part then-part sections of each rule. Therefore, a total number of (double the number of rules) neural networks are provided, where the neural networks respectively output data based on a plurality of rules. Owing to the provision of the if-part and then-part neural networks, this system can express complex input/output relationship. On the other hand, this system cannot clearly express the internal knowledge of the neural network. The expression of knowledge by a three dimensional data, e.g. x.sub.1, x.sub.2, and x.sub.3 makes it difficult to use a rule-knowledge owned by human being which is an object of a fuzzy theory or to extract rules from the neural networks. Furthermore, it is complex to suitably set up the plural neural networks. As described above, the decision made by the neural network is not fully generalized, and contain large errors against evaluation data.
Recently, a system for expressing the input/output relationship based on a fuzzy theory is proposed. That is, the proposed fuzzy system expresses human knowledge based on "if-then" rule. Unlike those strict numerical models, since the proposed fuzzy system expresses a variable with a membership function like "small" or "big", it can easily express human knowledge, and thus, usefulness of the system is highly evaluated. Nevertheless, a fuzzy theory involves difficulty to properly determine the membership function and establish fuzzy rules, and as a result, the membership function and the fuzzy rules are determined using a trial and error method.
In the light of the background mentioned above, it is desired to provide an improved neural network which is capable of effectively combining the capability to correctly express human knowledge according to a fuzzy theory and the training potential of the neural network itself. As a result of extensive study according to this object, some of practical methods have been introduced, which are described in the following documents.
In summary, the proposed fuzzy inference system contains following technical problems to be solved.
As described above, the conventional neuro-fuzzy system has drawbacks that the knowledge cannot be clearly expressed by a neural network, the structure of the system is complicated and the if-part and the then-part cannot be trained simultaneously since the if-part and then-part are formed of respective neural networks, and the generalization is insufficient.